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Art Unit: 2621 

DETAILED ACTION 



Continued Examination Under 37 CFR LI 14 

1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.1 14. Applicant's submission filed on 07/05/2006 has been entered. 

Claim Rejections - 35 USC §102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

3. Claims 1, 4-11, 13-22, and 24-36 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Merheim et al. (US 2002/0135483). 

Re claims 1, 22, 25, 26, 27, and 32, Merheim discloses a computer-readable medium 
comprising software (1 1 of fig. 2, Note computer code) for a video surveillance system (fig. 2), 
comprising code segments (11 of fig. 2, computer code) for operating the video surveillance 
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system based on video primitives, wherein the code segments for operating the video 
surveillance system comprise: 

code segments for identifying one or more event discriminators (100-120 of fig. 3; see 
also [0013] and [0016] and [0061], Note areas represent change of some kind that has occurred 
in the recorded image in comparison with the reference image; these changes can be that person 
has entered the monitored location or a tree is blowing in the wind in the monitored location); 

code segments for extracting video primitives (130-150 of fig. 3, [0061], Note when the 
areas have been extracted, an object is associated with each area in an object extraction step 
(140 of fig. 3) for easier management of the different areas. Instead of storing images of an area t 
selected area characteristics are stored, such as one or more of for example, coordinates in the 
image, size, outline, average intensity, circumference and intensity variations; see also figs. 4-6); 
and 

code segments for extracting event occurrences from the video primitives using at least 
one of the one or more event discriminators (160 of fig. 3 and 200, 210, and 220 of fig. 7; see 
also [0064]-[0065]) 

Re claim 4, Merheim further discloses code segments for archiving the extracted video 
primitives (140-150 of fig. 3). 

Re claims 5 and 24, Merheim further discloses comprising code segments for undertaking 
a response based on extracted event occurrences (170 of fig. 3, [0066]). 

Re claim 6, Merheim further discloses wherein the response comprises initiating another 
sensor system (12 of fig. 2). 
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Re claim 7, Merheim further discloses code segments for calibrating the video 
surveillance system ([0014], Note the monitoring system can be set up to classify the object as a 
human alarm object as soon as an object is determined to be a human object, but it can also be 
set up to classify the object as a human alarm object if the object is determined to be human and 
also fulfils some additional criterion, such as where in the image the human object is located). 

Re claims 8, 28, and 33, Merheim further discloses wherein the code segments for 
calibrating comprise code segments for self-calibrating the application-specific hardware the 
video surveillance system ([0014], Note the monitoring system can be set up to classify objects). 

Re claim 9, Merheim further discloses wherein the code segments for self-calibrating 
comprise: code segments for detecting as least one object in a source video (140, 150, and 160 of 
fig. 3) and code segments for tracking the object ([0064], Note by minimizing the matching 
difference for all objects at the same time, a good approximation is obtained of the object's 
previous history, which is known as tracking). 

Re claim 10, Merheim further discloses wherein the code segments for detecting at least 
one object comprise; code segments for detecting at least one object via motion of the object (fig. 
8, Note divergence image, motion object) and code segments for detecting at least one object via 
change in a background model ([0013]; fig. 8, Note background image). 

Re claim 11, Merheim further discloses wherein the code segments for self-calibrating 
comprise code segments for identifying trackable areas ([0009]); and code segments for 
identifying typical sizes of typical objects ([0010], Note classifying in each of the monitoring 
modules the object, based on characteristics, such as a characteristic of the type: size, shape 
and/or movement history, associated with the object, if the object is a human alarm object). 
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Re claim 13, Merheim further discloses code segments for tasking the video surveillance 
system with event discriminators (170 of fig. 3). 

Re claim 14, Merheim further discloses wherein the code segments for tasking comprise 
code segments for identifying at least one object (fig. 8, Note matching, the object's history). 

Re claim 15, Merheim further discloses wherein the code segments for tasking comprise 
code segments for identifying at least one spatial area ([0007]). 

Re claim 16, Merheim further discloses wherein the code segments for tasking comprise 
code segments for identifying at least one temporal attribute ([0036], Note date and time) 

Re claim 17, Merheim further discloses wherein the code segments for tasking identify at 
least one interaction (vector and speed of the object, [0027]). 

Re claim 18, Merheim further discloses wherein the code segments for tasking identify at 
least one alarm (170 of fig. 3, fig. 8, display of object of alarm operator). 

Re claim 19, Ito further discloses wherein the video primitives are from at least one of a 
video sensor (7 of fig. 2) and another sensor (12 of fig. 2). 

Re claim 20, Merheim further discloses wherein the video primitives (140 of fig. 3) are 
retrieved from an archive of video primitives (100 of fig. 3). 

Re claim 21, Merheim further discloses a computer system comprising the computer- 
readable medium (1 1 of fig. 2, Note computer code) of claim 1. 

Re claims 29-31 and 34-36, Ito further discloses wherein event occurrences are extracted 
based on video primitives and non-video primitives ([0013], Note moving objects and stationary 
objects); wherein at least one event discriminator includes at least two of the following: an object 
(140 of fig. 3, see also fig. 8), a spatial area (120 of fig. 3, see also fig. 8), a temporal attribute 
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(date and time, [0036]), an interaction , and an alarm (170 of fig. 3); wherein at least one event 
discriminator defines an interaction between one or more video primitives (150 and 160 of fig. 3, 
see also fig. 8), between one or more spatial areas of interest, and/or between one or more 
temporal areas of interest. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 1, 4-22, and 24-36 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Araki et al. (US 4,737,847) in view of Ito et al. (US 6,404,455). 

Re claims 1, 12, 21-22, 25, 26, 27, and 32, Araki et al disclose a video surveillance 
system (figs. 87 and 88) for performing the information processing steps, comprising code 
segments for operating the video surveillance system based on video primitives (fig. 46), 
wherein the code segments for operating the video surveillance system comprise: 

code segments for identifying one or more event discriminators (SETTING AREA of fig. 
88; 296, 297, 299, and 300 of fig. 46; col. 19, line 55-col. 20-line 8); 

code segments for extracting video primitives (PICTURE PROCESSING of fig. 88; See 
also 291 of fig. 46, Note object extraction and tracking, fig. 2); and 
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code segments for extracting event occurrences (INTRUDER DISCRIMINATION of fig. 
88, See also 292 of fig. 46) from the video primitives (291 of fig. 46) using at least one of the 
one or more event discriminators (299, 300, and 297 of fig. 46). 

It is noted that Araki et al suggest while the present invention shall now be described with 
reference to the preferred embodiments shown in the drawings, it should be understood that the 
intention is not to limit the invention only to the particular embodiments shown but rather to 
cover all alterations, modifications and equivalent arrangements possible within the scope of 
appended claims. However, Araki et al do not particularly disclose a computer readable medium 
encoded software comprising code segments for calibrating comprise code segments for manual 
calibration; code segments for semi-automatic calibration and code segments for automatic 
calibration as claimed. 

Ito teaches a computer readable medium (10 of fig. 1) encoded software comprising code 
segments for operating the CPU (6 of fig. 1), code segments for calibrating comprise code 
segments for manual calibration (col. 12, lines 21-25); wherein code segments for semi- 
automatic calibration and code segments for automatic calibration (col. 12, lines 26-29, Note the 
foregoing description about the system for processing the tracking picture of the entering object 
is concerned with the process of enabling the pan and tilt head control unit (8 of fig. 1) to output 
a zooming control signal for controlling the zoom lens (16 of fig. 1) and thereby automatically 
magnifying the image on the screen. Alternatively, the zooming control signal manually 
controlled by the observer may be inputted to the pan and tilt head control unit 8 through the 
external VF (1 1 of fig. 1). This is a manual adjustment of the magnified image). 
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Therefore, taking the teachings of Araki and Ito as a whole, it would have been obvious 
to one of ordinary skill in the art to incorporate the teachings of Ito into the system of Araki in 
order to provide a picture processing system for tracking an entering object by automatically 
detecting the entering object and magnifying the detected entering object, for allowing an 
observer to more easily make sure of the entering object on a video monitor screen. 

Re claim 4, Araki et al further disclose code segments for archiving the extracted video 
primitives (OPERATION & INTRUDER (POSITION, SIZE, and SPEED) of fig. 88; 291c of 
fig. 46). 

Re claims 5 and 24, Araki et al further disclose further comprising code segments for 
undertaking a response based on extracted event occurrences (QUESTION & ANSWER of fig. 
88, 292 of fig. 4). 

Re claim 6, Araki et al further disclose wherein the response comprises initiating another 
sensor system (136 of fig. 19, Note alarm sensors). 

Re claim 7, Araki et al further disclose code segments for calibrating the video 
surveillance system (INTRUDER or OBJECT: POSITION, SIZE, and SPEED of fig. 28; see 
also fig. 2, Note object extraction and frame tracking). 

Re claims 8, 28, and 33, Araki et al further disclose wherein the code segments for 
calibrating comprise code segments for self-calibrating the application-specific hardware (291 of 
fig. 46, see also fig. 2) the video surveillance system. 

Re claim 9, Araki et al further disclose wherein the code segments for self-calibrating 
comprise: code segments for detecting as least one object in a source video (291c of fig. 1, 
wherein the picture processing is described in fig. 2, SUBSTRACT, FILTERING, SLICING of 
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fig. 2), and code segments for tracking the object (FRAME TRACKING of fig. 2; Note this 
picture processing procedure is executed for each input picture frame, and thus processed object 
is subjected to a frame tracking and is provided, together with a warning level value later 
described in the monitoring zone, to an abnormality discrimination means (12 of fig. 1 or 22 of 
fig. 4) of a unique arrangement in the present invention, see also 438 of fig. 70 ). 

Re claim 10, Araki et al further disclose wherein the code segments for detecting at least 
one object comprise; code segments for detecting at least one object via motion of the object 
(INTRUDER: position, size and speed of fig. 88); and code segments for detecting at least one 
object via change in a background model (SETTING AREA of 88, Note setting up different 
background model). 

Re claim 1 1, Araki et al further disclose wherein the code segments for self-calibrating 
comprise code segments for identifying trackable areas (438 of fig. 70); and code segments for 
identifying typical sizes of typical objects (INTRUDER: SIZE of fig. 88). 

Re claim 13, Araki et al further disclose code segments (INTRUDER 
DISCRIMINATION of fig. 88, 291c and 292 of fig. 46; 1 1 and 12 of fig. 4) for tasking the video 
surveillance system with event discriminators. 

Re claim 14, Araki et al further disclose wherein the code segments for tasking comprise 
code segments for identifying at least one object (col. 5, lines 47-53). 

Re claim 15, Araki et al further disclose wherein the code segments for tasking comprise 
code segments for identifying at least one spatial area (predetermined area, SETTING AREA of 
fig. 88, fig. 47). 
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Re claim 16, Araki et al further disclose wherein the code segments for tasking comprise 
code segments for identifying at least one temporal attribute (151e, TIMER, of fig. 22) 

Re claim 17, Araki et al further disclose wherein the code segments for tasking identify at 
least one interaction (fig. 47). 

Re claim 18, Araki et al further disclose wherein the code segments for tasking identify at 
least one alarm (298 of fig. 46, see also figs. 5, 5, 8, 10). 

Re claim 19, Araki et al further disclose wherein the video primitives are from at least 
one of a video sensor (290 of fig. 46) and another sensor (136 of fig. 19). 

Re claim 20, Araki et al further disclose wherein the video primitives are retrieved from 
an archive of video primitives (224 of fig. 32). 

Re claims 29-31 and 34-36, Araki et al further disclose wherein event occurrences are 
extracted based on video primitives and non-video primitives (fig. 3); wherein at least one event 
discriminator (12 of fig. 1, 22 of fig. 4) includes at least two of the following: an object 
(INTRUDER of fig. 88), a spatial area (SETTING AREA of fig. 88), a temporal attribute (fig. 
22), an interaction (fig. 71-74), and an alarm (28 of fig. 4); wherein at least one event 
discriminator (22 of fig. 4) defines an interaction between one or more video primitives, between 
one or more spatial areas of interest (26-27 of fig. 4, fig. 5 for warning levels), and/or between 
one or more temporal areas of interest (Timer of fig. 22). 

Conclusion 

6. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 
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Pavilidis et al. (US 2003/0053659) discloses moving object assessment system and 
method. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tung Vo whose telephone number is 571-272-7340. The 
examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on 571-272-7418. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571- 272-1000 



Contact Information 




Tung Vo 
Primary Examiner 
Art Unit 2621 



